This invention relates to online systems and in particular to reviewing advertisements in an online system.
A large number of online systems derive revenue by showing advertisements to their users. Advertisements allow advertisers, or other providers, to obtain public attention to their products, services, opinions, or causes. Presenting advertisements allows advertisers to persuade persons viewing the advertisement to respond in a certain way to advertised products, services, or other information. Providers advertising their products, services, opinions, or causes in the online system may manually create an advertisement through a web page provided by the online system or automatically create an advertisement through an application programming interface (API).
As the online system grows and the number of users that visit the online system increases, the number of providers trying to advertise their products in the online system also increases. Additionally, online systems often specify certain policies for advertisements to comply with before being presented by the online system. Manual review of advertisements for compliance with online system policies is a time consuming process that is not scalable as the number of providers promote their products increases. Additionally, manual review of advertisements may lead to inconsistencies in review as reviewers may have different views regarding compliance of an advertisement.
Another problem with manual reviews is that advertisers do not know the likelihood a newly created advertisement will be approved. Accordingly, a large number of newly created advertisements are submitted for review with a high probability or rejection. Many rejected advertisements are modified and re-submitted multiple times, further increasing the number of reviews by the online system.